Method and apparatus for coding information

ABSTRACT

A method for inserting coded information into an image comprising the steps of analyzing the image, identifying the edge regions within the image, determining the strength of the edge region, and producing an edge function which is dependent on the strength of the edge. The edge function is added to the image in order to represent coded information.

BACKGROUND AND SUMMARY OF THE INVENTION

The present invention relates to a method and apparatus for theinsertion, and subsequent decoding, of coded information into images.

It is known to insert codes into images for example video transmissionsor video clips or stills transmitted across a telecommunication link,for the purpose of identifying the owner of the images. There are anumber of known schemes for inserting identification codes into the syncperiods, and more recently it has been proposed to insert identificationcodes into the image itself, but in such a manner that the code cannotbe detected by the eye.

WO 95/14289 discloses the embedding of an identification code throughoutan image by modulating a digitised version of the image with a smallnoise signal. The specific system described suffers from thedisadvantage of requiring to have to hand the original image for codeidentification. Further improvements in code robustness for transmissionover telecommunications links or broadcasting are also desirable.

WO 95/20291 discloses a method of hiding copyright related messages witha digital. data work, which relies on commonly occurring patterns orsequences of data in the work acting as signposts to target dataelements which are modified according to certain rules. The disclosedmethod suffers from a lack of robustness to signal degradation.

The above references suffer from a disadvantage that they are concernedwholly or principally with the digital domain, and the disclosedtechniques are not suited to the analog domain, in particular wheredigital to analog conversion and analog to digital conversion may easilylose or degrade individual pixel values.

In our copending application PCT/GB96/00246 (the national phase entry ofwhich is U.S. application Ser. No. 08/875,546, filed Jul. 31, 1997),there is disclosed and claimed a method for inserting coded informationinto an image, comprising analysing the image, identifying stronglyfeatured regions and inserting coded information into these regions.Because the method relies on an analysis of the entire image and codebeing inserted in strongly featured regions, rather than in individualpixels, as in the prior art, the code is better able to survive analogto digital conversions and digital to analog conversions, where therewill inevitably be pixel misalignments between the original andprocessed images, and hence the pixel values will be apparently altered.

By “strongly featured regions” is meant regions of primary strength towhich the eye responds in viewing an image, for example, texturedregions or lines or boundaries between two regions of differentluminance. In such regions, it is possible to insert a relatively largeamount of information without significantly altering the image in itsappearance to the eye. Edge regions between areas of different luminanceare preferred since these are very distinctive and will permit codeinsertion without visibly degrading the image, and the coded informationis inserted into such strongly featured regions by altering thestructure of the image in such region in a predictable or identifiablemanner. The image is altered by applying an insert function to an areaincluding the strongly featured region, which insert function graduallydecreases in intensity from the centre of its region of application, soas to blend with the surrounding region.

A main advantage of applying the coded information by altering the imagein a predictable or identifiable manner, is that the information can berecovered in a decoding process at a remote location without having theoriginal to hand.

As disclosed in PCT GB96/00246 the image, for example, a video frame orfield, is divided up into a number M×N of blocks in M rows and Ncolumns, each block comprising nxn pixel elements (e.g. 8×8). Stronglyfeatured regions are searched in each block for insertion of code. Inorder to encode a significant amount of information into an image, it isnecessary to apply the insertion function to a number of edges in theimage. If for example, one edge is chosen in each block into which theimage is divided, then in order to increase confidence when attemptingto recognise the code, the edges in one row may be encoded according totwo separate pseudo-random codes, representing a “1” or “0”.

Prior to inserting the information, the image is assessed as to whetherthe image is suitable for code insertion, for example, the degree ofconsistency of the edge, the definition of the edge centre to preventinsertions when the image is determined to be unsuitable. Problems havearisen in the method described in PCT/GB96/00246 where more than onedistinct edge is present in the image, or the part of the image underconsideration for code insertion; for example there may be a cornerfeature.

An object of the present invention is to provide a method of embeddingcoded information into images by adding into the image a function whichis dependant on the edge activity in the image.

The present invention accordingly provides in one aspect a method forinserting coded information into an image, comprising the steps ofanalysing the image, identifying edge regions calculating the strengthof the detected edge and producing an edge function dependent on thestrength of the edge, and adding the edge function to the image in orderto represent coded information.

In accordance with the present invention, an image (or block within animage) may be selected to be encoded using an edge detection basedscheme, the selection being dependant on the edge activity in the block.This allows for encoding of blocks with multiple features such as blockscontaining a corner or block containing two edges. In these blocks, anedge detection scheme is applied to the block to produce a functionwithin the block which has a value at each pixel, which is related tothe strength of the edge at that pixel. This edge function is suitablyscaled, and optionally multiplied by a circular function centred on thecentre of the block. The scaled edge function is then added to the imageon a pixel by pixel basis, with the sign of the function, plus or minus,representing a “1” or “0” bit of information. In a decoding operation,edges are detected and an estimate is made of the appropriate edgefunction. This estimate is then correlated with the image in order toreveal the existence of coded information.

In another aspect, the invention provides a method of decodinginformation contained in an image, the method comprising analysing theimage, identifying edge regions and providing an estimate of the edgefunction dependent on the strength of the edge, and correlating theestimated edge function with the image in order to identify codedinformation present within the image.

As preferred, the image may be divided into M×N blocks, and anencoding/decoding operation is carried out on each block. Further, inaccordance with the invention, an assessment is made of the type offeatures present in the image or image block, and a coding function isadded to the image, the type of function depending on the type of image,as more fully described in our Application PCT/GB96/00246. Thus if onlya single main edge is present, then an elliptical function may be addedcentred on the edge, if no distinct feature is present, then a circularfunction may be added, and if more than one distinct edge is present,then the edge function of the present invention may be added.

Another aspect of the present invention concerns synchronising thedecoding process. An encoded image may be presented to the decoder in ageometrically altered state from which it was encoded. The geometrictransformation may be for example an offset (x,y) translation, or ascaling in one or both axes (including different scaling in each axis),or a rotation. If the geometric transformation is of a significant size,then in order to decode it, this geometric distortion must be accountedfor. The decoder synchronising process achieves this by an analysisprocess involving splitting the image into blocks, analysing the blocksfor the type of insert and decoding the block accordingly. The processproduces a measure of coded insets and their centres and the regularityof their structure. The offsets, scaling and rotations are thencalculated from the geometric variations of the centres and structure.This is then used to correct the image before the final decoding.

BRIEF DESCRIPTION OF THE DRAWINGS

A preferred embodiment of the invention will now be described withreference to the accompanying drawing wherein:

FIGS. 1 to 3 are diagrams for explaining the preferred method of thepresent invention.

FIG. 4 is a flowchart illustrating the operation of the invention.

DESCRIPTION OF THE PREFERRED EMBODIMENT

The encoding and decoding algorithms of the preferred embodiment arelisted below as a sequence of steps:

Encoding Algorithm

1) split the frame into M*N adjacent blocks each of n*n pixels

2) calculate the local orientation for each pixel in each block

3) calculate the edge detection function for each block.

4) calculate the dominant orientation for each block, the amount ofactivity in each block, and the consistency of the dominant orientation

5) to encode one data bit process each block within a predefined set ofblocks distributed throughout the image, as follows:

5a) look up the next element of the PRS (psuedo random sequence) for thedata bit

1) if it is a 1 set to add

2) if it is a−1 set to subtract

5b) segment the blocks in the following categories

a) a single edge/line in a dominant orientation

b) a low activity block

c) several major lines or edges

5c) process the blocks as follows:-

a) add/subtract an elliptical function

a1) centred on the edge

a2) aligned to the block orientation

a3) scaled by the activity in the block

b) add/subtract a circular function

b1) centred on the centre of the block

b2) fixed strength

c) add/subtract the edge detection function

c1) centred on the centre of the block, and modulated by a circularfunction.

c2) scaled to bring it within the magnitude range of the ellipticalfunction in a).

6) repeat step 5) until all the bits are encoded.

Decoding Algorithm

1) split the frame into M*N adjacent blocks each of n*n pixels

2) calculate the local orientation for each pixel in each block

3) calculate the edge detection function for each block.

4) calculate the dominant orientation for each block, the amount ofactivity in each block, and the consistency of the dominant orientation

5) to decode one data bit process each block within the predefined setof blocks distributed throughout the image, as follows:-

5a) segment the blocks in the following categories

a) a single edge/line in a dominant orientation

b) a low activity block

c) several major lines or edges

5b) process the blocks as follows:-

a) calculate the concave/convex-ness of an elliptical function

a1) centred on the edge

a2) aligned to the block orientation

a3) scaled by the activity in the block

b) calculate the concave/convex-ness of a circular function

b1) centred on the centre of the block

b2) fixed strength

c) calculate the concave/convex-ness of an edge detection function

c1) centred on the centre of the block and modulated by a circularfunction

c2) scaled to bring it within the magnitude range

5c) correlate the convex/concave-ness of the block with the next elementof the data bit 1 PRS and accumulate the result.

5d) correlate the convex/concave-ness of the block with the next elementof the data bit 0 PRS and accumulate the result.

6) compare the data bit 1 accumulated correlation with the data bit 0accumulated correlation. The larger of the two is the decoded data bit.The size of the correlation is the confidence in the result.

7) repeat steps 5) and 6) for each data bit.

Referring now to FIGS. 1 to 3, the frame axis conventions used todescribe the mathematics are shown in FIG. 1. A pixel is defined by itscoordinates (x,y) and its luminance value is r(x,y). Note that the topleft pixel of an image is the (0,0) pixel, and that the y axis has itspositive direction down the frame.

As shown in FIG. 2, the frame is segmented into non-overlapping blocks,each block being of size n by n pixels. Smaller values of n mean that itis more likely that only a single edge will be found in any given block.It also means that more individual blocks will be available for thecorrelation process. Larger values of n mean that larger, and thereforemore robust, insertions can be made. In practice a good choice of n is8. The block axis conventions used in segmenting the frame are shown inFIG. 2.

Referring to FIG. 3, the local orientation for each point in the frameis calculated from four surrounding points by a process as describedbelow. This gives a vector at each point in the frame, with themagnitude of the vector representing the strength of the feature, andthe angle representing twice the local orientation. This is illustratedin FIG. 3. Thus the local orientation gives a measure of gradient ofluminance in a particular direction within the block. A very large valueof orientation indicates the existence of an edge; In this double angleform, the vectors can be vector-averaged over a block, to give the localorientation for the block. This provides a relatively fast estimatingalgorithm.

e.g., as shown in FIG. 3:

θ˜−45 degrees: by convention, θ is associated with point a=(x,y)

−dx˜0.7

−dy˜−0.7

θ is estimated from

a=r(x,y)

b=r(x,y+1)

c=r(x+1,y)

d=r(x+1,y+1)

e=d−a

f=b−c

Re=−2*e*f

Im=e²−f²$\theta = {\frac{1}{2}{{Tan}^{- 1}( \frac{Im}{Re} )}}$

θ is in single angle form

Re,Im are in double angle form

orientations are averaged in the Re,Im double angle form.

In the present invention, the measurement of luminance gradient isemployed not only to give an assessment of the type of image in theblock, and for determining the location of edges within the block asdisclosed in PCT/GB96/00246, but also to determine the value of the edgedetection function which is to be added to the image information inorder to insert coded information within the image.

The edge detection function is calculated by taking the magnitudes ofthe orientation-vectors for each pixel, then for each pixel the averageof the magnitudes of it's neighbours is calculated. I.e. for pixels (a)to (t) arranged in 4 rows of 5 columns, thus: ${\begin{matrix}a & b & c & d & e \\f & g & h & i & j \\k & l & m & n & o \\p & q & r & s & t\end{matrix}\quad \begin{matrix}{{the}\quad {averages}} \\{{{are}\quad {calculated}}\quad} \\{{as}\quad {shown}}\end{matrix}\quad {\begin{matrix}0 & 0 & 0 & 0 & 0 \\0 & {( {a + b + f + g} )/4} & {( {b + c + g + h} )/4} & {( {c + d + h + i} )/4} & \quad \\0 & {( {f + g + k + l} )/4} & {( {g + h + l + m} )/4} & {( {h + i + m + n} )/4} & \quad \\0 & {( {k + l + p + q} )/4} & {( {l + m + q + r} )/4} & {\cdots \quad {{etc}.}} & \quad\end{matrix}\quad}}\quad$

The average magnitude for each pixel is then multiplied by a circularfunction centred on the center of the block. The edge detection functionhas a single number per pixel in the block but with the outer pixels setto zero. This is to say, that in the example discussed above, the edgedetection function for pixels (a) to (e), (f), (j), (k), (o) and (p) to(t) are set to zero. This gives a pattern of values for the edgefunction thus:

0 0 0 0 0 0 0 0 0 0.0 0.0 0.0 0.2 0.0 0.0 0 0 0.1 0.2 0.0 0.6 0.1 0.0 00 0.1 0.1 0.0 0.9 0.0 0.1 0 0 0.1 0.2 0.0 0.7 0.0 0.1 0 0 0.1 0.2 0.00.1 0.0 0.1 0 0 0 0 0 0 0 0 0

The edge detection function is scaled to get it in the range 0.0 to 1.0,so that it is in the same range as an elliptical function calculated forblocks having just a single dominant edge as described inPCT/GB96/00246. This is done by taking the magnitude at each pixel,scaling it by a fixed factor, and limiting it between zero and a fixedfactor.

The scale edge detection function is then added to the original image.

What is claimed is:
 1. A method for inserting coded information into animage, comprising the steps of analysing the image, identifying edgeregions, determining the strength of the edge region and producing anedge function which is dependent on the strength of the edge, and addingthe edge function to the image in order to represent coded information.2. A method according to claim 1, further comprising the step ofdividing up the image into M*N blocks of pixels, and performing theaforesaid steps in each block of the image.
 3. A method according toclaim 1, further comprising the steps of dividing the image into aplurality of blocks of m×n pixels (where m and n are any whole number),calculating orientation vectors for each pixel in each block andcalculating the average mangitude of the vector for each selected pixelby calculating the average of the vector magnitudes of a plurality ofpixels immediately adjacent the selected pixel, scaling the averagemagnitude of the vector for each selected pixel to produce a scaled edgefunction, and adding the scaled edge function to the image to representcoded information.
 4. A method according to claim 3 wherein the averagemagnitude of the vector for each selected pixel is multiplied by acircular function centred on the centre of each block to produce an edgefunction, the edge function is scaled to produce a scaled edge function,and the scaled edge function is added to the image to represent codedinformation.
 5. A method according to claim 2, further comprising thesteps of dividing the image into a plurality of blocks of m×n pixels(where m and n are any whole number), calculating orientation vectorsfor each pixel in each block and calculating the average magnitude ofthe vector for each selected pixel by calculating the average of thevector magnitudes of a plurality of pixels immediately adjacent theselected pixel, scaling the average magnitude of the vector for eachselected pixel to produce a scaled edge function, and adding the scalededge function to the image to represent coded information.
 6. A methodaccording to claim 5 wherein the average magnitude of the vector foreach selected pixel is multiplied by a circular function centered on thecenter of each block to produce an edge function, the edge function isscaled to produce a scaled edge function, and the scaled edge functionis added to the image to represent coded information.
 7. A method ofdecoding information contained in an image, the method comprisinganalysing the image, identifying edge regions and providing an estimateof the edge function dependent on the strength of the edge, andcombining the estimated edge function with the image in order toidentify in a correlation operation coded information present within theimage.
 8. A method according to claim 7, comprising the step of dividingup the image into M*N blocks of pixels, and performing the aforesaidsteps in each block of the image.
 9. A method according to claim 7,further comprising the steps of dividing the image into a plurality ofblocks of m×n pixels (where m and n are any whole number), calculatingorientation vectors for each pixel in each block and calculating theaverage magnitude of the vector for each selected pixel by calculatingthe average of the vector magnitudes of a plurality of pixelsimmediately adjacent the selected pixel, scaling the average magnitudeof the vector for each selected pixel to produce a scaled edge function,and adding the scaled edge function to the image to represent codedinformation.
 10. A method according to claim 8, further comprising thesteps of dividing the image into a plurality of blocks of m×n pixels(where m and n are any whole number), calculating orientation vectorsfor each pixel in each block and calculating the average magnitude ofthe vector for each selected pixel by calculating the average of thevector magnitudes of a plurality of pixels immediately adjacent theselected pixel, scaling the average magnitude of the vector for eachselected pixel to produce a scaled edge function, and adding the scalededge function to the image to represent coded information.
 11. A methodaccording to claim 9, wherein the average magnitude of the vector foreach selected pixel is multiplied by a circular function centered on thecenter of each block to produce an edge function, the edge function isscaled to produce a scaled edge function, and the scaled edge functionis added to the image to represent coded information.
 12. A methodaccording to claim 10, wherein the average magnitude of the vector foreach selected pixel is multiplied by a circular function centered on thecenter of each block to produce an edge function, the edge function isscaled to produce a scaled edge function, and the scaled edge functionis added to the image to represent coded information.
 13. A method ofdecoding information contained in an image, the method comprisinganalysing the image, identifying strongly featured regions, determiningfor at least one such region an anticipated insertion of codedinformation, and combining such anticipated insertion with the image todetermine in a correlation operation whether there has been insertedinto the strongly featured region coded information, wherein in order totake account of geometric transformation of the image to be decoded, aninital decoding operation is carried out to determine the periodicityand/or regularity of the anticipated insertions, whereby to determineany such geometric transformation.
 14. A method of decoding according toclaim 13, comprising the steps of splitting the image into blocks,analysing the blocks for the type of insert, decoding the blocks toproduce a measure of coded inserts and their centres and the regularityof their structure, calculating offsets, scaling and rotation from thegeometric variations of the centres and structure and correcting theimage prior to decoding the embedded codes.